Neurobiology of Language

Sentence comprehension requires the incremental construction of syntactic structure and semantic interpretation. Prior neural work (Nelson et al., 2017) identified key neural events at major phrase boundaries during sentence comprehension. To investigate a behavioral correlation of these processes, we used self-paced reading to examine the impact of syntactic phase boundaries, semantic congruence, and sentence structure on sentence processing. Participants read object-relative, subject-relative, and canonical control sentences one word at a time and a subsequent comprehension task. Reading times were analyzed relative to phrase boundaries, node-closing operations, and semantic congruence. Object-relative sentences produced the greatest processing difficulty, demonstrated by increased reading times and decreased comprehension accuracy. Reading times peaked at the phrase boundaries, indicating that processing costs are tied to constituent completion rather than individual lexical categories. Reading times also increased with the number of syntactic constituents completed at a phrase boundary. Agent-patient semantic congruence produced its largest effects in object-relative sentences, suggesting that semantic information interacts with syntactic computations when processing demands are greatest. These findings demonstrate that self-paced reading is sensitive to the incremental processing associated with syntactic constituent completion. Processing costs are tied more closely to phrase completion than to individual lexical categories, scale with the amount of syntactic structure completed at a boundary and interact with agent-patient semantic interpretation during object-relative sentence comprehension. Together, these findings support a view of sentence comprehension in which syntactic structure building and semantic interpretation proceed incrementally and interact continuously throughout online language processing.

Successful speech perception requires listeners to bin continuous acoustic information into discrete phonetic categories. However, some people maintain within-category acoustic information (gradient) while others discard category-irrelevant information (discrete) during perception. Listeners also vary in how consistently they label speech sounds and more gradient/consistent labeling has been linked with better speech-in-noise (SIN) perception. Here, we test how neuroanatomical properties of the brain's major speech-language and auditory pathways relate to individual differences in speech categorization and SIN processing. We measured phonetic categorization and SIN comprehension via phoneme labeling and QuickSIN tasks. Diffusion-weighted imaging (DWI) with probabilistic tractography estimated axonal density within the bilateral arcuate fasciculi and brainstem-cortical auditory projections. Anatomical morphology (surface area, gray matter volume, thickness) was also quantified in the adjacent frontotemporal cortical areas and midbrain. Behaviorally, we found more consistent categorizers had better performance on the QuickSIN. DWI showed that more gradient listeners had greater white matter density in the left arcuate fasciculus and brainstem-cortical auditory pathways, while better SIN performance was predicted by denser white matter in the brainstem-cortical auditory pathways. Morphometric results revealed more consistent listening was associated with greater cortical thickness in right superior temporal gyrus and more gradient listening was associated with greater surface area in right pars opercularis. We infer that individual differences in phonetic categorization relate to SIN comprehension and are at least partially explained by neuroanatomical properties of the auditory-linguistic brain.

Neural decoding can be viewed as a representation learning problem in which neural activity is mapped into an intermediate representation before downstream reconstruction. The choice of intermediate representation influences both performance and learning difficulty. Here we developed a novel framework for studying how intermediate representation choice influences downstream learning and reconstruction. As a proof-of-concept, we instantiated our framework using diffusion latent representations extracted from different diffusion timesteps for neural speech decoding. Component-wise evaluation showed that reconstruction performance differed substantially across diffusion timesteps, with teacher-forced Word Error Rates of 44.7%, 7.5%, and 3.5% for different latent models. These results demonstrate that diffusion latent representations can serve as effective intermediate representations for learning from neural activity, but that their effectiveness depends strongly on the selected diffusion timestep. More broadly, our framework provides a basis for systematically studying how intermediate representation choice influences downstream learning and reconstruction.

Classic neurobiological models of human speech and language have emphasized the dominant role of temporal lobe in speech perception, while premotor regions including the ventral premotor cortex (PMv) are situated at the level of articulatory processing. However, accumulating evidence from neuroimaging, clinical, and computational studies suggests that premotor cortex may contribute to speech processing beyond articulation. The precise extent and functional organization of these speech-related representations, however, remain unclear. In this study, we combined naturalistic speech perception with computational encoding models to characterize the organization of speech representations within PMv. We functionally localized a cortical region that encompasses previously described premotor speech areas, which we term the premotor language area (PML). Using acoustic, phonemic, semantic, and deep neural speech representations, we found that PML contains representations spanning the full speech-processing hierarchy, from low-level acoustic features to high-level semantic information. These representations are arranged along a smooth posterior-anterior gradient, with increasingly abstract speech representations emerging toward anterior PML. Moreover, this organizational gradient mirrors the canonical speech processing hierarchy in the temporal auditory regions. These findings challenge the traditional view of premotor cortex as primarily an acoustic-articulatory unit, and instead identify PML as a hierarchically organized speech-processing region that parallels the temporal auditory cortex. This provides a new framework for understanding the role of premotor cortex in speech perception.

While evidence suggests magnocellular deficits in the geniculostriate pathway in adults with dyslexia, neural deficits in the subcortical pathways during childhood remain unclear. Here, we used high-resolution fMRI to investigate subcortical abnormalities in Chinese children with developmental dyslexia. Fast achromatic motion stimuli and slowly drifting chromatic gratings were used to assess magnocellular (M) and parvocellular (P) functions, respectively. Relative to controls, children with dyslexia showed a selective reduction in responses to the M stimulus in the ventromedial pulvinar (vmPul) and the superficial layers of the superior colliculus (SCs), along with significantly reduced SCs-vmPul connectivity. Importantly, while vmPul responses to the M stimulus were positively associated with reading skills in healthy controls, this correlation was absent in children with dyslexia. Unlike previous findings in adults, the lateral geniculate nucleus (LGN) exhibited a non-selective reduction in responses to both stimuli, no volume reduction, and no correlation with reading ability. These findings demonstrate a selective deficit to achromatic motion processing in the colliculus-pulvinar pathway in children with dyslexia, which contributes to their reading difficulties. This early subcortical disruption differs from, and precedes, the neural deficits previously reported in the adult LGN, offering new insight into the developmental trajectory of dyslexia.

Speech consists of regularities at different timescales. Already during infancy, neural electrophysiological activity aligns to these rhythms. The degree to which infants exhibit neural tracking of speech can be linked to their language development. In this study, we examined how the neural tracking of sung speech develops across age, from infancy to early childhood, and across different frequency bands (i.e., at the stress, syllabic, and phonemic rates), and whether neural tracking at each frequency and age predicts childrens language outcomes. We included 2565 children of the longitudinal YOUth cohort. Children listened to Dutch sung nursery rhymes while EEG was recorded at three measurement waves. After preprocessing the data, we included 955 children at 5 months, 1048 children at 10 months, and 795 children at 2-4 years. The final sample consisted of 750 children who also completed a receptive vocabulary test at 2-4 years. Children from 5 months onwards showed significant neural tracking of stressed syllables, syllables, and phonemes, measured with speech-brain coherence (SBC). Unexpectedly, there were no developmental changes in SBC across different frequency bands from infancy to early childhood. As expected, children with larger receptive vocabularies showed increased SBC in the stressed syllable rate. These findings suggest that stronger tracking of stressed syllables is related to individual differences in language ability.

Building upon foundational psychological theories of event segmentation, this study addresses the limitation of overreliance on temporal boundaries as the primary segmentation criterion. Drawing on two experiments of direct and indirect causation in Mandarin Chinese, this study demonstrates how cognitive segmentation granularity and semantic integration jointly shape syntactic encoding. Results reveal distinct event encoding patterns for direct and indirect causation: coarse-grained segmentation leads to compact syntactic structures (e.g., verb-resultatives), while fine-grained segmentation yields varied multi-clausal expressions. Chinese speakers update event models via prediction errors of intentionality and protagonists, and tend to establish event boundaries at goal-relevant action endpoints when construing causal chains. These conceptual dimensions exert a modulating influence on both event segmentation and semantic integration. We propose a triad model integrating event segmentation, semantic integration, and linguistic specificity, providing a unified framework for elucidating the mind-language interface in conceptual construction and event coding of causation.

Contextual word learning is a fundamental mechanism for vocabulary acquisition during childhood. In adults, successful inference of word meaning from context is intrinsically rewarding, and is associated with greater enjoyment and greater activity in reward-related brain regions. Whether similar reward mechanisms support word learning in children, and whether they differ as a function of ability, remains unknown. We used functional magnetic resonance imaging (fMRI) to examine neural responses during contextual word learning in 25 children aged 11-13 years with typical reading skills and in 20 age-matched children with dyslexia. Neurotypical readers showed enhanced activation in core reward-processing regions, including the ventral striatum, when successfully learning the meanings of novel words. In contrast, children with dyslexia did not exhibit comparable reward-related responses despite performing the same task. Crucially, this group difference was specific to word learning, as no significant group differences were observed in ventral striatal responses during a non-linguistic monetary reward task. In addition, to confirm the behavioural relevance of these neural findings, we examined an age-matched, independent sample of children. We found that stronger reading skills were associated with greater enjoyment during successful word learning. Together, these results suggest that interactions between reward and language systems during contextual word learning is influenced by reading proficiency. Reduced intrinsic reward responses to successful language learning may contribute to differences in reading development and have implications for the design of more engaging and effective reading interventions for struggling readers.

We present the BCCWJ-Brain dataset, a multi-modal neuroimaging resource comprising functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and electroencephalography (EEG) data recorded from native Japanese speakers reading newspaper articles from the Balanced Corpus of Contemporary Written Japanese (BCCWJ). Neural data were collected from 112 participants (36 fMRI, 35 MEG, and 41 EEG) as they read twenty newspaper articles presented in a Rapid Serial Visual Presentation (RSVP) paradigm. By providing three complementary neuroimaging modalities collected under identical naturalistic reading stimuli, this dataset provides a cognitive benchmark for computational models such as large language models. The dataset is publicly available on the OpenNeuro platform, offering a valuable resource for neuroscience, natural language processing, and related research fields.

Large language models trained on next-word prediction have impressive linguistic capabilities. This suggests that the goal of temporal prediction is essential to language processing, but how this goal impacts the structure of speech representations in the human brain remains unknown. Here, we test the hypothesis that prediction is facilitated by the temporal straightening of representational trajectories along the speech processing hierarchy. We developed a methodology for measuring the curvature of these trajectories using fMRI. Our method exploits a previously unknown connection between the timescale of single-unit responses and the curvature of population trajectories. We examined brain responses of subjects listening to natural speech. Response trajectories were most curved in lower-level auditory areas and progressively straightened along the cortical hierarchy. We presented the same speech stimuli and perturbed versions thereof to wavLM---a speech representation model that is well aligned with human brain responses---and found that hierarchical straightening effects are strongest for stimuli whose statistical structure resembles natural speech. Together, our results establish a direct connection between the goal of temporal prediction, the geometry of neural speech representations, and the cortical hierarchy of representational timescales.

Prosody is a fundamental aspect of speech characterized by suprasegmental features such as pitch. Prosodic pitch contours are used to convey speakers intentions, for example, to make a statement or ask a question. Understanding these intentions requires abstracting continuous, variable pitch information into discrete categories. Category perception has been proposed to recruit the motor system in an effector-specific manner, whereby cortical areas controlling motor effectors support speech sound recognition by identifying articulatory gestures. However, it remains unclear whether effectors involved in pitch production similarly contribute to prosodic category perception. To address this question, we collected magnetoencephalography data from 29 participants (15 females) while they first sang pitches arranged in five-tone melodies and then identified the prosody (Statement vs. Question) of single words varying in pitch contour along a five-level continuum. Using a region-restricted searchlight approach to decode singing from rest, we localized two premotor regions for pitch production, corresponding to the ventral and dorsal laryngeal motor cortex (LMC). A separate neural decoding analysis revealed that perceived prosodic categories were decodable in these regions, especially from the dorsal LMC that is more closely associated with pitch regulation. Importantly, decoding performance mirrored behavioral discriminability of prosodic categories across the continuum, suggesting that these regions are involved in perceptual decision-making. Finally, pitch motor areas exchanged category-related information with auditory regions, indicating these areas do not merely echo the processing in auditory regions. Together, these findings highlight effector-specific motor support for prosodic category perception, thereby broadening our understanding of motor involvement in speech perception. Significance StatementSpeech perception has been proposed to recruit the premotor cortex, with different subregions linking speech sounds to the articulatory gestures used to produce them. We investigated this idea through prosody--pitch changes in speech conveying meanings such as statements and questions. Using magnetoencephalography, we identified pitch motor areas during a singing task and tested whether they represent perceived prosodic categories. We found that prosodic categories were distinguishable in these regions and that this neural discriminability mirrored behavioral discriminability across clear and ambiguous prosody, suggesting involvement in perceptual decision-making. These findings are unlikely to reflect passive echoes from auditory regions, as pitch motor areas actively influenced them during categorical processing. Our results highlight effector-specific motor support for forming abstract prosodic representations.

Understanding speech in noise is a central challenge of everyday communication, yet listeners often succeed by using prior context. How the brain uses such context remains debated: it may refine predictions about upcoming words, or it may provide a higher-level framework that helps degraded speech cohere into meaning. Here we combined simultaneous EEG-fNIRS recording with hierarchical multivariate encoding models to track how prior context shapes speech processing from acoustics to words and sentential meaning. Participants listened to natural spoken narratives under clear speech, noisy speech, and context-supported noisy speech conditions. Context brought comprehension of noisy speech close to clear-speech levels. EEG revealed that contextual support reduced neural encoding of lexical surprisal and entropy, indicating weaker tracking of local word-level prediction demands. In contrast, when context was available, fNIRS showed enhanced encoding of sentence-level semantic integration across frontal regions and the right angular gyrus, and stronger angular gyrus encoding predicted better comprehension. By combining EEG and fNIRS to capture complementary electrophysiological and hemodynamic signals, this multimodal approach reveals a hierarchical shift in degraded speech comprehension: prior context does not simply improve word-by-word prediction, but scaffolds the integration of noisy input into coherent discourse.

Listeners discretize the speech signal by assigning sounds to phonetic categories, though there is variability in how individuals accomplish categorization. Having more consistent categorization of sounds may be advantageous for understanding speech-in-noise (SIN). Though, it is unclear how different levels of neural processing in the auditory system reflect these perceptual differences. We recorded brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) while listeners actively labeled vowels along an acoustic-phonetic continuum using a visual analog scale. We computed intertrial consistency of neural responses to index the stability of listeners' neural speech representations across stimulus presentations. We also assessed how faithfully midbrain and cortical responses represented stimulus acoustics using representational dissimilarity matrices (RDMs) computed across all token pairs. Neural RDMs were then compared with acoustic and phonetic category RDMs to assess whether FFRs and ERPs carried gradient vs. categorical information of the speech signal. We found greater behavioral consistency during phoneme labeling was correlated with improved SIN scores. Neurally, we found greater cortical or subcortical consistency predicted greater behavioral consistency. RDMs revealed subcortical responses retained more acoustic details, while cortical responses more closely reflected abstract phoneme categories. Our findings reveal important benefits of perceptual consistency to other domains of speech perception. We find perceptual consistency is driven by more consistent encoding of speech at either a cortical or subcortical level. More consistent sensory processing could provide a more stable readout of the speech signal to higher cortical brain areas which could confer advantages to later perceptual processes downstream.

The visual word form area (VWFA) is a hallmark of literacy in the human brain. Alongside reading acquisition, the functional organization of the ventral temporal cortex (VTC) undergoes substantial changes. However, it remains unclear what factors uniquely drive the emergence and continued development of the VWFA, or more broadly shaping category selectivity across the VTC. Here, we combined cross-sectional and longitudinal data from children in early childhood (3-9 years) to investigate the development of visual language selectivity. We found that after controlling for age, literacy acquisition drove increases in word selectivity of the VWFA, but not the continued development of other early-developed category selectivity. Sensitivity to spoken higher-level language information within the VWFA was also associated with reading ability. By projecting the VWFA defined at the later time point onto each childs earlier time point, we also found that the pre-VWFA showed no preferential tuning to any particular visual category. Finally, longitudinal changes within the VWFA, both increased responses to visually presented words and decreased responses to auditory control conditions, were associated with changes in functional connectivity of VWFA to high-level language regions, even after controlling for initial activity levels. Together, the current study shows a unique role for literacy acquisition and experience-dependent connectivity changes in the emergence and functional specialization of the VWFA, providing empirical evidence for the revised neuronal recycling hypothesis and connectivity hypothesis of functional brain organization.

Language production requires the coordination of multiple cognitive processes. The ability to anticipate and override a habitual response in favor of a contextually-appropriate response are key subprocesses of cognitive control which enable speakers to communicate effectively. Word retrieval involves the co-activation of semantically related alternatives from which the speaker must select the appropriate target representation. Although cognitive control mechanisms have been proposed to contribute to resolving semantic interference during language production, the nature of these control processes remain unclear. Studies investigating the temporal dynamics of cognitive control during decision making tasks have led to a distinction between two operating processes: proactive control, initiated prior to the occurrence of conflict, and reactive control recruited after conflict is detected. We investigated the roles of proactive and reactive control in resolving interference between competing linguistic representations during word retrieval. We analyzed congruency sequence effects combined with delta-plot distributional analyses to dissociate potential adjustments in proactive versus reactive cognitive control in a picture-naming task manipulating semantic context compared to a minimally-linguistic Stroop-like paradigm. Reaction time distributional properties following semantically related trials revealed the engagement of proactive control in semantic interference resolution during word retrieval in the PWI task. In contrast, reactive inhibitory control was engaged in resolving semantic interference following low conflict trials. This distinction was not present in the minimally-linguistic task, which did not appear to engage adaptive control to the same extent. These findings demonstrate that both proactive and reactive cognitive control mechanisms contribute to language production, and are engaged dynamically, adjusting trial-by-trial to resolve semantic interference during word retrieval. In addition, our study provides important insight into the comparison of language with other cognitive domains and positions linguistic paradigms as being instrumental in the study of cognitive control dynamics.

The human brain rapidly transforms continuous speech into structured, meaningful linguistic representations, yet how prior knowledge constrains this process remains unclear. To characterize this influence, we combined MEG recordings acquired during audiobook listening with corpus-derived transition probabilities over syntactic features defined within both phrase-structure and dependency-based grammars. Across grammatical formalisms, prior knowledge selectively sharpened the neural representation of memory-related features indexing syntactic structures that must remain open for future completion. This enhancement was strictly local: immediately preceding contexts improved neural decoding at both word onset and offset, whereas longer histories produced either a return to baseline at the word level or a deterioration in decoding performance. By contrast, integration-related features indexing the completion of syntactic operations showed no benefit from prior knowledge and were represented most strongly at word offset, consistent with their dependence on word-level structural resolution. These dissociable dynamics reveal two concurrent neural computational regimes for syntactic processing: a forward-looking, locally maintained predictive code for pending structure and an integrative code engaged when structure is resolved. More broadly, our findings impose a mechanistic constraint on neural theories of language processing and on accounts that equate prediction in human language comprehension with the comparatively unconstrained operations of large language models (LLMs).

Audiovisual semantic matching can be achieved using either written words or pictures, yet whether these formats engage shared semantic matching representations with similar temporal dynamics remains unclear. We recorded electroencephalography from 27 participants while they performed audiovisual semantic matching tasks in which spoken words were paired with either written words or pictures. Stimuli included both natural and man-made objects. Time-resolved multivariate pattern analyses (MVPA or decoding), cross-decoding, and temporal generalization analyses were used to characterize the temporal dynamics of semantic processing. Reliable decoding of matching versus mismatching judgments emerged in both word and picture conditions. Decoding onset that significant above chance level occurred earlier for written words than for pictures and cross-decoding analyses revealed successful generalization between word and picture formats. Temporal generalization analyses further demonstrated distinct representational dynamics across formats, with word processing characterized by predominantly time-specific neural representations and picture processing showing more sustained and temporally stable representations. In addition, matching-related discrimination emerged earlier for natural objects than for man-made objects across both formats. The results suggest that speech-word matching shows earlier neural evidence of audiovisual alignment than speech-picture matching, potentially reflecting differences in how auditory linguistic input is integrated with visual information across representational formats.

The ability to derive neural-level language coding models holds great scientific and clinical potential. Current approaches are limited by the scale and ethological validity of input data; applications requiring large, rare, or naturalistic samples in particular would benefit from the ability to infer neural coding from incidental everyday speech. Here we present a novel pipeline designed to leverage spontaneous and incidental naturalistic speech. This pipeline performs transcription, segmentation, and video-assisted diarization, as well as alignment and spike detection of neural data. We apply this pipeline to a dataset derived from 21 patients (6+ days each, over 800 hours and 5 million words total). We benchmark both encoding and decoding models against extensive and rare ground-truth control datasets consisting of human-curated word-level temporal alignment and manually sorted spikes. We further validate our approach by quantifying representational drift, effect of dataset size, and differences between six brain areas. Together, these findings demonstrate that incidental natural speech is sufficiently processed in the brain to enable the estimation neural-level embeddings.

How the brain constructs meaning across extended contexts remains poorly understood. While neural responses to words and sentences are well characterized, much less is known about the brain mechanisms supporting narrative comprehension. Sentence-level studies suggest that neural activation increases as word meanings are integrated into sentence meaning. At the discourse level, theories propose that narratives depend on situation models, possibly engaging networks beyond core language regions, including the default mode network. Because narrative comprehension unfolds over longer timescales, processing time may be a bottleneck. In this MEG study, we tested how representation size and presentation rate shape neural responses by varying linguistic structure (words, sentences, stories) and the speed of visual text in 1-4-word chunks. We found an early bilateral story effect in visual cortex, followed by a spatiotemporal progression of activity along the temporal lobes that culminated in a three-way contrast among word lists, sentence lists, and stories. Faster presentation altered this pattern: the left-lateralized story effect disappeared, and the right-lateralized effect became more spatially restricted. Under Fast presentation, significant effects were limited to left lateral language cortex distinguishing coherent inputs from word lists, and to two right-hemisphere story effects in extended language regions. We also observed a context effect in the Slow Story condition, with neural responses remaining constant as the narrative unfolded while they increased in the SentenceList and WordList conditions. This effect was absent under Fast presentation, suggesting story-specific comprehension that is temporally constrained. Together, the findings identify temporal constraints as a key determinant of the neural signatures of narrative processing.

Reported perceptual differences in autism may arise from reduced use of prior context to shape incoming sensory input. Speech perception provides a critical test of this account because stable perception requires listeners to integrate variable acoustic signals with contextual expectations. This study examined context-dependent modulation of speech encoding in autistic and non-autistic adults using the frequency-following response (FFR), a neurophysiological measure of phase-locked auditory encoding. Participants heard English intonational pitch contours presented in repetitive and variable contexts while EEG was recorded. Principal component analysis of FFR metrics yielded components indexing neural encoding fidelity and timing. Non-autistic participants showed enhanced encoding fidelity in more predictable contexts, whereas autistic participants showed reduced context-dependent modulation. Neural encoding timing also showed divergent context effects across groups, suggesting altered balance between feedback-based predictive mechanisms and locally driven adaptation processes. Within the autistic group, greater context-related modulation of encoding fidelity was associated with lower ADOS-2 Social Affect severity but poorer speech-in-noise perception, suggesting that the functional impact of contextual modulation depends on input reliability and task demands. These findings indicate that context-dependent modulation of speech encoding is altered in autism and may contribute to individual differences in auditory and social-communicative function.